The state of the art in combining real world imagery with additional imagery from another source is a process that requires considerable precision. One of the most common applications is to combine images generated by a computer with images acquired from a traditional motion picture, video, or digital camera. In order to seamlessly combine the images, the computer generated image must be created using a virtual camera and lens parameters that closely match the optical parameters of the physical camera.
In this type of application, there are two main sets of parameters that must be measured. The external parameters, such as the physical camera's position and orientation in the physical world, are typically called the extrinsic parameters. These parameters can be directly measured using a variety of methods. The other set of parameters are the internal lens optical parameters that typically cannot be measured without disassembling the lens, which is typically undesirable or impossible.
Internal lens optical parameters are typically called the intrinsic parameters, which include the lens' focal length, the optical center of the projected image on the image sensor, and the level of distortion imparted by the camera lens. An additional desired optical parameter is the entry pupil location, which describes where incoming rays are focused to a point by the lens. The location of this point is very important for accurate rendering of a virtual image, as the origin of the virtual camera must be placed precisely where the entry pupil of the physical camera was located for the virtual image's perspective to match the physical image's perspective.
Many methods have been used to calibrate intrinsic lens parameters. These include photographing a 2-D or 3-D calibrated object of known size, and using the correlation between the known points of the calibrated object and the location of these points in the captured digital image to calculate the intrinsic lens parameters. All of these methods share some requirements: the physical size of the object must be known, and the physical positions of the measured object must occur in at least two separate planes in order to separate the effects of focal length (zoom) from the physical proximity of the camera to the object. This requirement is met automatically with a 3-D calibration object, but a 2-D calibration object must be photographed from a variety of angles to provide sufficient information. Typically, at least fifteen separate images of a 2-D calibration target are required for each lens adjustment setting to provide accurate results. Since an adjustable lens must be measured at many places through its adjustment range, this multiple frame requirement makes it very difficult to handle the large number of measurements needed to calibrate adjustable lenses. Additionally, both of these approaches have further problems. 3-D calibrated objects or fixtures of sufficient size and accuracy are difficult to make and store. The typical methods for lens calibration using planar objects are not able to calculate the entrance pupil location of the lens due to the lack of information about the relative position of the camera and object.
Accordingly, none of the above methods work well for the demands of motion picture and television production, which typically require very rapid equipment calibration and transportation.